Towards Generation Of Fluent Referring Action In Multimodal Situations

Referring actions in multimodal situations can be thought of as linguistic expressions well coordinated with several physical actions. In this paper, what patterns of linguistic expressions are commonly used and how physical actions are temporally coordinated to them are reported based on corpus examinations. In particular, by categorizing objects according to two features, visibility and membership, the schematic patterns of referring expressions are derived. The di erence between the occurrence frequencies of those patterns in a multimodal situation and a spoken-mode situation explains the ndings of our previous research. Implementation based on these results is on going.